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The  networking  of  combat  vehicle  simulators;  as  illustrated 
by  SIMNET,  provides  a  method  for  collective  training  that  supple¬ 
ments  field  exercises.  To  realize  the  training  potential  of  this 
"electronic  battlefield,"  trainers  must  be  provided  with  tools  to 
use  in  identifying  and  illustrating  key  events  during  postexer¬ 
cise  After  Action  Reviews  (AARs) .  As  a  first  step  in  meeting 
this  need,  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  sciences  (ARI)  developed  a  PC-based  Unit  Performance 
Assessment  System  (UPAS)  to  collect  data  on  vehicle  status  and 
firing  events  broadcast  over  the  simulation  network.  This  report 
describes  the  procedures  and  outcome  of  an  effort  to  design 
graphic  UPAS  AAR  aids  that  integrate  network  data  with  other 
sources  of  Information  to  provide  a  more  complete  picture  of  unit 
performance  than  is  possible  with  network  data  alone. 

This  research  has  been  used  as  input  to  develop  a  joint 
service  standard  for  distributed  interactive  simulation  that 
addresses  unit  performance  measurement  and  feedback  systems.  The 
software  implementation  of  the  UPAS  AAR  aids  designed  under  this 
project  are  undergoing  user  testing  by  trainers  at  the  Combined 
Arms  Tactical  Training  Center,  Fort  Knox,  Kentucky. 

The  work  described  in  this  report  is  a  portion  of  the  re¬ 
search  task,  Training  Requirements  for  Combined  Arms  Simulators. 
This  task  supports  a  memorandum  of  agreement  entitled  "The 
Effects  of  Simulators  and  Other  Resources  on  Training  Readiness," 
signed  16  January  1989.  Parties  to  this  agreement  are  the  U.S. 
Army  Training  and  Doctrine  Command,  the  U.S.  Army  Center  at  Fort 
Knox,  the  U.S.  Army  Materiel  Command,  and  ARI. 
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PLATOON-LEVEL  AFTER  ACTION  REVIEW  AIDS  IN  THE  SIMNET  UNIT 
PERFORMANCE  ASSESSMENT  SYSTEM 


EXECUTIVE  SUMMARY 


Requirement; 

The  networking  of  simulators  provides  a  means  of  conducting 
oolleotive  training  that  supplements  field  exercises.  To  take 
advantage  of  the  training  potential  of  networked  simulators,  the 
Army  just  ensure  that  units  receive  prompt  quality  feedback  dur¬ 
ing  postexeroise  After  Action  Reviews  (AARs) . 

A  previous  effort  by  the  U.S.  Array  Research  Institute  for 
the  Behavioral  and  Social  Sciences  produced  a  prototype  PC-based 
Unit  Performance  Assessment  System  (UPAS)  that  collects  data  on 
firing  events  and  vehicle  status  from  simulation  networking 
(SIMNET)  exercises,  loads  these  data  into  a  relational  database, 
and  provides  tools  for  analyzing  these  data  using  original  or 
predefined  graphs  and  tables.  The  prototype  UPAS  includes  a  plan 
view  display  that  replays  an  exercise  from  a  bird's-eye  view. 

The  goal  of  the  work  described  in  this  report  was  to  modify  and 
I  expand  the  prototype  UPAS  to  support  platoon-level  AAR  more 
I  effectively. 


Procedure: 

The  prototype  UPAS  was  assessed  in  terms  of  its  ability  to 
meet  four  criteria  for  a  performance  feedback  system  in  the 
SIMNET  environment:  capability  to  integrate  data  from  diverse 
sources;  capability  to  help  a  trainer  identify  and  illustrate  key 
exercise  events  quickly;  capability  to  support  multiechelon  AARs; 
and  flexibility.  Concepts  for  improving  the  plan  view  and  creat¬ 
ing  additional  AAR  aids  were  designed  and  implemented  in 
software. 


Findings: 

Roughly  36%  of  the  Armor  Platoon  Mission  Training  Plan 
standards  supported  by  SIMNET  require  integrating  the  network 
data  collected  by  the  UPAS  with  data  on  the  terrain  situation, 
unit  plans,  tactical  communications,  and  observable  behaviors  of 
soldiers.  We  identified  the  need  to  develop  AAR  aids  that  sup¬ 
port  the  integration  of  network  data  with  other  data  sources.  In 
addition,  the  prototype  plan  view  failed  to  meet  the  criterion  of 
supporting  rapid  identification  and  illustration  of  key  exercise 
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events  because  moving  from  one  point  In  time  of  the  exercise  to 
another  was  a  alow  process. 

Concepts  were  designed  to  improve  the  plan  view  and  imple¬ 
ment  three  new  types  of  AAR  aid  formats:  a  battle  flow  chart  to 
trace  a  unit's  movement  over  the  course  of  an  exercise,  battle 
snapshots  to  provide  static  bird's-eye  views  of  the  battlefield 
at  discrete  points  in  time,  and  an  exercise  timeline  to  provide 
information  about  temporal  relationships  among  the  application  of 
control  measures,  movement,  and  firing  events. 

The  new  plan  view,  battle  flow  chart,  and  snapshot  were  used 
to  integrate  terrain  and  planning  data  with  network  data.  The 
grid  map  display  for  these  aids  Includes  major  terrain  features 
and  unit  control  measures  from  the  unit's  operations  order.  A 
quick  search  capability  was  added  to  these  aids  to  allow  the  user 
to  move  quickly  to  points  of  interest  within  the  exercise. 

The  exercise  timeline  is  a  separate  display  that  indicates 
whan  a  unit  crosses  a  control  measure,  halts,  starts  moving, 
receives  indirect  fire,  first  receives  enemy  direct  fire,  sus¬ 
tains  a  casualty,  first  fires  on  the  enemy,  and  destroys  an  enemy 
vehicle.  Friendly  firing  events  cued  by  enemy  firing  events, 
compliance  of  movement  with  control  measures,  and  movement  rate 
are  performance  aspects  that  can  be  assessed  with  the  timeline. 

The  exercise  timeline  and  the  battle  flow  provide  means  of 
identifying  key  exercise  events  and  the  time  of  their  occurrence. 
Additional  Information  about  those  events  can  be  gained  from  the 
plan  view  or  battle  snapshots,  as  appropriate,  once  the  time  and 
nature  of  key  events  have  been  identified. 


Utilization  of  Findings: 

These  findings  will  be  used  to  ensure  that  the  UPAS  supports 
timely  and  effective  AARs  at  the  end  of  SIMNET  exercises.  They 
are  applicable  to  future  generations  of  networked  simulators, 
such  os  the  Close  Combat  Tactical  Trainer,  and  they  are  appli¬ 
cable  to  the  design  of  a  feedback  system  for  collective  embedded 
training.  Finally,  these  findings  will  support  the  conduct  of 
training  research  in  the  simulation  networking  environment,  in¬ 
cluding  the  development  of  improved  tools  for  controlling  the 
behavior  of  Semi-Automated  Forces  (SAFOR) . 
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PLATOON-LEVEL  AFTER  ACTION  REVIEW  AIDS 
IN  THE  SIMNET  UNIT  PERFORMANCE  ASSESSMENT  SYSTEM 

Introduction 

S-imulation  Networking  f SIMNET) 

The  networking  of  combat  vehicle  simulators  provides  a 
method  for  training  crews  to  work  together  as  part  of  a  unit  and 
training  units  to  work  together  as  part  of  a  larger  organization 
(U.S.  Army  Armor  School,  19G9a;  Thorpe,  1988).  Information 
produced  by  each  simulator,  such  as  its  location  on  the  terrain 
database  and  the  target  location  of  each  firing  engagement,  is 
broadcast  over  a  network  and  picked  up  by  other  simulators.  A 
computer  graphics  generator  with  each  simulator  is  able  to 
reconstruct  a  realtime  "out  the  window"  picture  of  the 
battlefield  using  broadcast  data  and  data  from  a  common  terrain 
database.  The  initial  application  of  networked  simulators, 
SIMNET,  was  developed  by  the  Defense  Advanced  Research  Projects 
Agency  and  included  simulators  for  armor  and  mechanized  infantry 
vehicles  (Thorpe,  1988) .  Rotary  wing  aircraft  were  subsequently 
added  to  the  SIMNET  family  in  a  project  referred  to  as  Airnet. 

The  goal  of  networking  simulators  is  to  support  collective 
training  rather  than  individual  skills  or  gunnery  training.  This 
restricted  goal  allows  SIMNET  to  avoid  certain  costly  options, 
such  as  a  high-resolution  graphics  generator.  For  example, 
while  SIMNET  may  lack  the  fidelity  required  to  use  it  to  train 
individual  gunnery  skills  (Drucker  and  Campshure,  1990) ,  it 
appears  to  be  capable  of  supporting  certain  collective  training 
requirements  such  as  training  a  unit  how  to  use  a  volume  of  fire 
to  cover  the  movement  of  another  friendly  unit  (Burnside,  1990) . 

The  cost  of  using  SIMNET  is  also  kept  low,  relative  to  that 
of  field-based  exercises,  by  employing  Semi-Automated  Forces 
(SAPOR) .  SAFOR  can  be  used  in  the  roles  of  threat  forces  or 
friendly  forces  (Mullally,  Petty,  and  Smith,  1991).  The  SAPOR 
function  allows  an  operator  to  control  a  large  number  of 
simulated  vehicles.  This  feature  offers  the  benefit  of  reducing 
the  training  support  personnel  costs  associated  with  collective, 
training  and  adds  further  to  the  overall  cost-effectiveness  of 
SIMNET. 


Purpose  of  Report 

This  report  describes  a  project  designed  to  develop  a  SIMNET 
Unit  Performance  Assessment  System  (UPAS)  to  help  trainers 
provide  units  with  feedback  during  platoon-level  After  Action 
Reviews  (AARs) .  This  report  describes 

•  the  AAR  concept; 

•  training  device  innovations  such  as  SIMNET,  that 
facilitate  application  of  the  AAR  concept; 
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•  the  need  for  a  product  like  UPAS  to  support  AARs  in  the 
SIMNET  environment; 

•  the  initial  prototype  UPAS; 

•  the  process  and  outcome  of  an  effort  to  define 
requirements  for  improving  the  UPAS  to  address  training 
requirements  more  effectively; 

•  UPAS  improvements  made  in  response  to  these  requirements; 

•  potential  applications  of  the  improved  UPAS  to  tasks  other 
than  supporting  SIMNET  AARs. 

The  After  Action  Review  (AAR)  Concept 

The  AAR  is  an  interactive  process  in  which  exercise 
participants  discuss  mission  planning  and  execution  under  the 
guidance  of  an  AAR  leader  (Scott,  1983;  Scott  and  Fobes,  1982; 
Meliza,  Sulzen,  Atwood,  and  Zimmerman,  1987;  Downs,  Johnson,  and 
Fallesen,  1987).  During  AARs,  exercise  participants  acquire 
information  about  how  to  improve  their  performance  through  the 
process  of  "discovery  learning." 

The  AAR  focuses  on  critical  events  having  a  direct 
influence  on  mission  outcome  (Scott,  1983;  Scott  and  Fobes,  1982; 
Meliza  et  al.,  1987),  rather  than  addressing  all  possible 
measures  of  performance  that  apply  to  an  exercise.  For  example, 
a  platoon's  mission  might  be  to  provide  covering  fire  for  a  unit 
conducting  an  assault.  If  the  platoon  does  not  provide  adequate 
covering  fire  during  the  assault,  then  it  has  failed  its  mission. 
Identifying  the  critical  events  that  led  to  this  failure  serves 
to  define  specific  corrective  actions.  Further,  linking  critical 
events  and  corrective  actions  to  mission  outcome  serves  to 
increase  their  credibility  in  the  eyes  of  exercise  participants. 

The  AAR  should  result  in  concrete  suggestions  for  correctirig 
performance  (Downs  et  al.,  1987).  During  an  AAR,  a  platoon  might 
find  that  it  was  late  in  providing  covering  fire  for  an  assault. 
Further  discussion  may  lead  to  the  conclusion  that  tardiness  in 
firing  was  due  to  the  fact  that  the  platoon  was  not  in  a  position 
to  observe  the  start  of  the  assault  and  the  unit's  plan  did  not 
provide  a  mechanism  for  coordinating  the  timing  of  suppressive 
fires  with  the  start  of  the  assault.  The  AAR,  in  this  case, 
points  the  way  to  specific  corrective  actions.  In  comparison,  if 
the  platoon  is  told  merely  that  it  failed  to  provide  covering 
fire,  then  corrective  actions  are  not  clearly  defined. 

In  preparing  for  an  AAR,  the  leader  identifies  critical 
events  that  made  substantial  contributions  to  mission  outcome  and 
diagnoses  unit  strengths  and  weaknesses  that  led  to  these  events. 
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During  the  AAR,  the  AAR  leader  guides  the  discussion  to  ensure 
that  key  events  and  their  causes  are  discussed.  Ideally,  the  AAR 
leader  will  have  AAR  aids  to  illustrate  exercise  events  and  their 
causes.  In  the  example  of  a  platoon  providing  covering  fire  for 
an  assault  element,  these  AAR  aids  might  include  a  graph  showing 
the  volume  of  fire  delivered  by  the  platoon  as  a  function  of  time 
and  a  figure  showing  intervisibility  between  the  platoon  and  the 
assault  force. 

Training  Device  Innovations  That  Facilitate  APPllcation__of  Jbhe 
AAR  Concept 

Three  major  innovations  in  training  devices  have  facilitated 
application  of  the  AAR  concept;  realistic  simulation  of  weapon 
effects,  instrumented  ranges,  and  the  networking  of  simulators. 
These  innovations  combine  to  facilitate  the  AAR  concept  by 

•  providing  a  credible  mission  outcome  around  which  an  AAR 
can  be  developed; 

•  employing  electronic  data  that  can  be  used  to  prepare 
AAR  aids;  and 

•  increasing  the  opportunity  for  units  to  train  by  reducing 
the  cost  of  training. 

Each  of  these  innovations  is  discussed  below. 

The  Army's  move  towards  realistic  simulation  of  weapon 
effects  in  force-on- force  exercises  began  with  REALTRAIN  (or 
Scopes)  and  continued  with  the  Multiple  Integrated  Laser 
Engagement  Simulation  (MILES)  .  REALTRAIN  employed  a  prcpcedure 
whereby  numbers  on  the  helmets  of  enemy  soldiers  or  vehicles  had 
to  be  identified  (using  scopes  affixed  to  weapons)  and  called  out 
by  the  firing  element  in  order  for  a  casualty  to  be  assessed. 
MILES  replaced  this  cumbersome  system  v/ith  eye-safe  lasers  and 
detector  belts,  but  REALTRAIN  was  used  to  test  the  training 
effectiveness  of  engagement  simulation  per  se.  Units  trained 
with  REALTRAIN  and  the  AAR  technique  performed  better  than 
conventionally  trained  units  in  terms  of  mission  accomplishment, 
casualty  exchange  ratios,  and  process-oriented  measures  of  team 
performance  (Meliza,  Scott,  and  Epstein,  1979;  Scott,  Meliza, 
Banks,  and  Hardy,  1979) .  The  use  of  engagement  simulation  helps 
to  ensure  that  there  are  credible  outcomes  of  training  exercises 
on  which  to  base  an  AAR. 

The  development  of  instrumented  ranges  was  the  second 
innovation  to  support  the  AAR  concept.  Instrumented  ranges 
collect  time-tagged  pof^ition  location,  firing  event,  and  casualty 
data  from  vehicles  on  a  near  continuous  basis.  The  U.S.  Army's 
National  Training  Center  (NTC)  at  Fort  Irwin,  California,  is  an 
instrumented  range  used  to  train  units  up  to  brigade  slice  level. 
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The  Precision  Range  and  Integrated  Maneuver  Exercise  (PRIME)  is 
an  instrumented  range  used  to  train  platoons  at  Fort  Hood,  Texas. 
The  position  location,  firing  event,  and  casualty  data  collected 
automatically  within  these  ranges  are  used  in  preparing  AAR  aids 
after  NTC  (Meliza  et  al.,  1987)  and  PRIME  exercises  (Witmer, 
1990).  The  benefits  of  these  ranges  can  be  better  appreciated 
when  one  considers  the  cost  of  trying  to  collect  these  time- 
tagged  data  using  trained  data  collectors.  Collecting  these  data 
during  the  REALTRAIN  validation  for  Armor/Anti -Armor  units 
required  a  data  collector  on  each  vehicle  in  the  exercise  and  a 
mapper  to  follow  each  unit  (Scott,  Meliza,  Hardy,  Banks,  and 
Word,  1979) .  Such  a  large  commitment  of  personnel  to  data 
collection  is  not  cost-effective. 

The  networking  of  simulators  was  the  third  innovation  to 
support  the  application  of  the  PiAR  concept.  SIMNET  offers  the 
greatest  potential  for  instrumented  data  collection,  because  the 
training  exercise  takes  place  in  an  electronic  environment. 

SIMNET  incorporates  the  first  two  innovations  supporting  AARS 
described  above,  and  goes  beyond  these  innovations  to  provide 
data  that  are  not  currently  collected  on  instrumented  ranges  in 
an  automated  fashion.  For  example,  SIMNET  makes  it  possible  to 
monitor  fuel  levels,  ammunition  levels,  weapon  system 
orientation,  and  engine  speed  throughout  an  exercise. 

In  order  to  make  use  of  more  realistic  weapons'  effects  and 
electronic  data  to  facilitate  AARs,  units  must  be  first  given  the 
opportunity  to  train.  The  high  cost  of  training  using 
operational  equipment,  combined  with  severe  limitations  in  the 
availability  of  training  resources,  acts  to  reduce  opportunities 
for  unit  tactical  training.  In  addition  to  providing  realistic 
weapons'  effects  and  electronic  data  to  prepare  AAR  aids,  SIMNET 
offers  the  advantage  of  addressing  problems  in  training  costs  and 
resource  availability.  Since  SIMNET  does  not  employ  operational 
equipment,  it  provides  a  savings  in  terms  of  fuel  expenditures 
and  reduced  wear  and  tear  on  operational  equipment.  Further, 
SIMNET  requires  fewer  personnel  to  support  training  exercises, 
because  SAFOR  can  be  used  to  play  the  role  of  both  threat  forces 
and  friendly  forces.  Finally,  the  capability  of  SIMNET  to  employ 
terrain  databases  from  a  wide  variety  of  training  areas  addresses 
the  need  for  more  varied  terrain  and  larger  training  areas  that 
face  units  when  conducting  unit  tactical  training  at  their  home 
station. 
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The  Prototype  Unit  Performance  Assessment  System 


SIMNET  has  powerful  tools  for  observing  replays  of  unit 
performance  during  AARs  (Thorpe,  1988) .  These  tools  include  a 
"Stealth  Vehicle"  that  provides  an  "out  the  window  view"  of  the 
action  from  any  point  on  the  battlefield  and  a  Plan  View  Display 
that  allows  the  action  to  be  observed  from  a  bird's-eye  view. 
However,  use  of  these  tools  requires  knowing  when  critical  events 
occurred  in  order  to  navigate  through  the  replays  effectively. 

In  addition,  problems  arise  in  using  a  single  Plan  view  and 
Stealth  to  host  AARs  when  multiple  exercises  are  conducted  at  the 
same  time,  or  when  exercises  are  conducted  at  company  level  and 
above.  The  latter  situation  creates  a  problem  in  that  AARs 
should  be  conducted  for  each  echelon  participating  in  the 
exercise,  one  at  the  highest  unit  level  and  one  for  each 
subordinate  unit. 

ARI  recognized  the  need  for  low  cost,  personal  computer- 
based  (PC-based)  methods  for  analyzing  data  from  SIMNET  exercises 
and  initiated  development  of  the  UPAS  (White,  McMeel,  and  Gross, 
1990) .  Figure  1  provides  a  graphic  summary  of  the  components  of 
the  prototype  UPAS.  The  system  collects  data  packets  from  the 
simulation  network  and  uses  these  packets  to  drive  a  replay  of 
the  exercise  from  a  bird's-eye  view.  The  system  also  extracts  a 
copy  of  the  data  in  the  packets  and  loads  these  data  into  a 
relational  database  management  system,  once  these  data  are  in 
the  database,  they  can  be  analyzed  using  Structured  Query 
Language  or  SQL  (XDB  Systems,  Inc.,  1990).  Graph  and  table 
editors  within  the  UPAS  can  be  used  to  create  a  menu  of  graph 
options  and  a  menu  of  table  options.  Each  of  the  major 
components  of  the  UPAS  are  described  below. 

Network  Data  Packets  and  Their  CQll_ectiQD__by.  the  JJPM 

UPAS  collects  virtually  all  of  the  data  packets  broadcast 
over  the  simulation  network  regarding  vehicle  status  and  firing 
events  and  loads  these  packets  into  a  raw  data  file.  The  types 
of  time-tagged  packets  are  described  below. 

•  The  Vehicle  Appearance  Packet  provides  the  information 
needed  to  simulate  continually  the  position  and  appearance 
of  each  vehicle  in  SIMNET.  The  information  contained  in 
this  packet  includes  the  vehicle  ID,  bumper  number,  a  force 
identifier  (1  for  BLUFOR  and  2  for  REDFOR) ,  the  elevation  of 
the  vehicle,  the  turret  azimuth,  the  gun  elevation,  the 
appearance  (live  or  burning) ,  the  type  of  vehicle,  the 
location  of  the  vehicle,  the  speed  of  the  vehicle,  the  speed 
of  the  engine,  and  the  direction  of  the  vehicle.  Figure  2 
illustrates  what  one  of  these  packets  looks  like  when  viewed 
through  the  UPAS  Packet  Display  feature. 
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Figure  1.  Overview  of  major  components  of  the  prototype  UPAS. 

•  The  Vehicle  Status  Packet  contains  information  about  a 
vehicle  other  than  that  required  to  simulate  the  outward 
appearance  of  the  vehicle.  This  information  includes:  the 
volume  of  fuel  in  each  of  the  vehicle's  tanks;  the  total 
number  of  ammo  rounds  available;  the  odometer  reading;  the 
type  of  vehicle;  a  rating  of  the  mobility,  fire  power,  and 
communications  status  of  the  vehicle  as  operational  or  non- 
operational . 

•  The  Fire  Packet  identifies  the  target  (if  known),  the  type 
of  ammunition  used,  the  firing  vehicle  ID,  the  location  of 
the  gun  muzzle,  the  type  of  fire,  the  number  of  rounds 
fired,  and  the  rounds  fired  per  second. 

•  The  Indirect  Fire  Packet  identifies  the  location  of  an 
indirect  fire  mission  impact,  the  result  of  the  impact,  the 
type  of  ammunition  employed,  and  the  number  of  rounds  fired. 
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•  Impact  Packets  describe  the  ground  impact  or  vehicle 
impact  resulting  from  a  firing  event.  These  packets 
indicate  the  location  of  the  impact,  the  distance  from  the 
muzzle  to  impact,  the  number  of  rounds,  and  the  number  of 
rounds  per  second,  the  ID  of  the  firing  vehicle,  and  the 
type  of  ammunition  used.  If  the  packet  describes  a  vehicle 
impact,  it  also  includes  the  ID  of  the  target  vehicle, 

•  The  Status  Change  Packet  describes  changes  in  the  status  of 
a  vehicle  resulting  from  damage  by  a  direct  or  indirect 
firing  event.  This  packet  identifies  the  vehicle  whose 
status  if  changed,  the  vehicle  causing  the  change,  and  the 
nature  of  the  change. 


VEHICLE  APPEARANCE 

VEHICLE  IDENTIFICATION;  00002/00096/00001 

EXEFtCISE;  42 

PROTOCOL;  Simulation 
VERSION:  3 

PDU8IZE:  144 

TIME;  10:03:23.64 

Bumper  MarKIng: 

Force:  2 

Capabilities; 

Vehicle  Elevation ;  24 1 . 70 

Turret  Azimuth:  6260  (MILS) 

Qun  Elevation:  18  (MILS) 

Appearance:  Live 

Vehicle  Class:  Tank 

Vehicle  Location:  ES91 657477 

Vehicle  Type  (1):  USSRT72M 

Vehicle  Type  (2) :  US  M 1 

Vehicle  Speed:  o.O  (Km/h) 

Engine  Speed: 

Direction:  308I  fMILSl 

<F1  >  to  track  vehicle,  ESC;  quit 

<F2>  Go  to  speolflo  record.  - > :  Next 

PgUp/PgDn  to  move  ahead  to  different  packet  type.  <[ _  :  Previous 

Record  #:  1 

Figure  2 .  Sample  Vehicle  Appearance  Packet  from  a  SIMNET 
exercise. 

A  SIMNET  exercise  is  capable  of  generating  a  large  number  of 
packets.  For  example,  a  short  ten  minute  exercise  with  firing 
events  may  generate  thirty  thousand  or  more  packets.  Most  of 
these  packets  are  Vehicle  Appearance  Packets,  because  one  of 
these  packets  is  generated  by  each  vehicle  at  one  second 
intervals. 
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The  raw  data  file  can  be  used  to  replay  the  mission  or 
critical  segments  of  the  mission  from  a  bird's-eye  view  using  a 
Plan  View  Display  as  illustrated  in  Figure  3.  Tactical  vehicles 
are  represented  by  rectangular  icons  for  which  weapon  system 
orientation  is  indicated  by  a  line  representing  the  gun  tube. 
Friendly  blue  force  (BLUFOR)  vehicles  are  represented  by  blue 
rectangles,  and  enemy  red  force  (REDFOR)  vehicles  are  represented 
by  red  rectangles.  Each  time  a  vehicle  fires,  it  brightens  in 
color  briefly.  Vehicles  that  become  casualties  change  color 
permanently  (blue  to  cyan  and  red  to  white) .  The  exercise  is 
replayed  over  a  grid  map  lacking  terrain  features. 

The  prototype  Plan  View  allows  the  user  to  magnify  the 
battlefield.  The  first  screen  displayed  after  initiating  the 
Plan  View  covers  an  area  16  kilometers  by  8  kilometers.  The  user 
has  the  option  of  limiting  the  area  covered  by  the  display  to  an 
area  1  kilometer  square,  so  that  the  area  is  magnified  by  a 
factor  of  128.  Intermediate  levels  of  magnification  may  also  be 
used. 


<F1>  to  Chongo  Soeie,  <F2>  to  Chanoo  VIoepoft,  <F3>  to  Chiiigo  Ttnio. 
<F5>  PtH  <F«>  Noxt  Event,  ProM  ESC  to  letum  «  PievioiM  Olaptoy 


Figure  3.  Prototype  Plan  View  Display. 


8 


The  Plan  View  is  linked  to  a  feature  called  a  Mission  Event 
List  that  helps  the  user  to  navigate  through  the  replay  of  an 
exercise.  A  Mission  Event  List  screen  allows  the  user  to  type  in 
key  time-tagged  events  from  the  unit's  operations  order,  as 
illustrated  in  Figure  4.  During  the  replay  of  the  exercise  using 
the  Plan  View,  the  user  can  press  the  <F6>  key  to  move  to  the 
next  time-tagged  event  in  sequence.  In  addition,  the  user  can 
move  to  a  particular  point  in  time  in  an  exercise  by  pressing  the 
<F3>  key  and  typing  in  the  new  time  to  which  the  user  wants  to 
move. 


Master  Event  List 


Event  - 

Time 

Move  out  of  assembly  area 

06:30 

Cross  Line  of  Departure 

06:45 

Cross  Phase  Line  Dog 

07:10 

Reach  Assauit  Position  Falcon 

07:30 

<F1  >  Save  Change  and  Exit  <F4>  Append 

<P2>  Edit  <F9>  Delete 

<F3>  Insert  < ESC >  Exit  Without  Change 

Figure  4.  Mission  Event  List  screen  for  loading  data  on  time- 
tagged  events  from  the  unit's  operations  order. 

Data.  Converalon  to  a  Relational  Database 

The  UPAS  contains  an  utility  that  converts  the  raw  data  and 
loads  them  into  a  relational  database.  Once  the  data  are  in  this 
database,  they  can  be  analyzed  by  non-programmers  using 
Structured  Query  Language  (SQL) . 

The  design  of  this  database  is  patterned  after  the  NTC 
Archive  database,  and  thus  it  is  referred  to  as  the  SIMNET/NTC 
database.  In  terms  of  the  number  of  data  tables,  the  SIMNET/NTC 
database  is  a  subset  of  the  NTC  Archive  database.  For  example. 
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the  NTC  database  contains  a  Minefield  Casualty  Assessment  Table 
that  is  not  included  in  UPAS,  because  minefields  are  not  played 
in  SIMNET.  On  the  other  hand,  certain  of  the  SIMNET/NTC  Tables 
in  UPAS  contain  information  that  is  unique  to  SIMNET.  Tables  1 
and  2  indicate  the  information  contained  in  two  tables  common  to 
the  two  databases.  Ground  Position  Location  Table  and  Paired 
Event  Table.  Items  with  an  asterisk  are  unique  to  SIMNET.  Other 
SIMNET/NTC  tables  are  described  in  White  et  al.  (1990). 

TABLE  1.  CONTENTS  OF  THE  SIMNET/NTC  GROUND  PLAYER  POSITION 
LOCATION  TABLE 

•  TIME  OF  VEHICLE  STATUS  UPDATE 

•  PLAYER  BUMPER  NUMBER 

•  LOGICAL  PTAYER  NUMBER 

•  POSITION  OF  VEHICLE  EXPRESSED  IN  TERMS  OF  X-Y-Z  COORDINATES 

•  POSITION  OF  VEHICLE  EXPRESSED  IN  X-Y  COORDINATES  RELATIVE 
TO  THE  ORIGIN  OF  THE  TERRAIN  DATA  BASE  * 

•  VEHICLE  SPEED  * 

•  VEHICLE  DIRECTION  * 

•  GUN  ELEVATION  * 

•  TURRET  AZIMUTH  * 

•  ENGINE  SPEED  * 

•  ODOMETER  READING  * 

•  TOTAL  AMOUNT  OF  AMMUNITION  * 

•  AMOUNT  OF  FUEL  LEFT  IN  VEHICLE  * 


TABLE  2.  CONTENTS  OF  THE  SIMNET/NTC  PAIRED  EVENT  TABLE 

•  TIME  OF  FIRING  EVENT 

•  TARGET  PLAYER  BUMPER  NUMBER 

•  TARGET  LOGICAL  PLAYER  NUMBER 

•  RESULT  OF  FIRING  EVENT  (MISS,  HIT,  OR  KILL) 

•  FIRING  PLAYER  BUMPER  NUMBER 

•  FIRING  PLAYER  LOGICAL  PLAY’IR  NUMBER 

•  TYPE  OF  WEAPON  SYSTEM  FIRED 

•  type  of  ammo  USED 

f  AN  INDICATION  OF  WHETHER  THE  EVENT  IS  A  FRATRICIDE  * 

•  POSITION  OF  TARGET  VEHICLE  EXPRESSED  IN  X,Y  COORDIN.ATES 
t  POSITION  OF  FIRING  VEHICLE  EXPRESSED  IN  X,Y  COORDINATES 

•  RANGE  OF  THE  ENGAGEMENT 

There  are  at  least  two  benefits  to  employing  a  common 
database  design.  First,  this  commonality  supports  efforts  to 
validate  SIMNET  training  against  NTC  training,  and  it  makes  it 
possible  to  develop  software  tools  for  analyzing  unit  performance 
in  one  environment  and  transfer  these  tools  to  the  other- 
environment.  Second,  the  NTC  database  is  designed  to  include  all 
of  the  combined  arms  elements.  As  more  elements  are  added  to 
SIMNET  or  future  generations  of  Army  networked  simulators,  the 
UPAS  database  will  readily  accommodate  these  additions. 
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To  facilitate  the  analysis  of  data  in  the  SIMNET/NTC 
database,  UPAS  also  contains  menu-based  table  and  graph  editors. 
Those  editors  are  used  in  combination  with  SQL  to  create  menus  of 
tabular  and  graphics  options  for  use  in  examining  unit 
performance.  Once  a  new  table  or  graph  has  been  "defined"  using 
these  editors,  its  name  is  added  to  the  menu  of  tables  or  graphs 
available  to  all  users  of  the  UPAS.  When  a  specific  graph  or 
table  option  is  selected,  the  UPAS  automatically  prepares  the 
table  or  graph  using  the  exercise  files  currently  being  examined. 
Using  these  menus  does  not  require  knowledge  of  SQL. 

The  graph  and  table  options  can  be  used  by  trainers  or 
researchers  to  analyze  unit  performance,  and  they  can  by  used  as 
audiovisual  aids  in  providing  feedback  to  exercise  participants 
during  AARs.  Table  3  shows  a  data  summary  table  that  can  be 
produced  using  UPAS,  and  Figure  5  illustrates  one  of  the  graphs 
produced  from  the  menu  of  graph  options.  Note  that  the  graph 
display  gives  the  user  the  option  of  changing  the  scale  of  the 
graph.  This  feature  is  important,  for  example,  when  the  user 
wants  to  focus  attention  on  the  volume  of  fires  during  a  specific 
period  of  time  during  the  exercise  (e.g.,  during  the  first  five 
minutes  after  initial  contaot  with  the  threat) . 


TABLE  3. 

FIRING  EVENTS 

DURING  SIMNET 

EXERCISE  AS  A 

FUNCTION  1 

UNIT  SIDE, 

,  TIME,  RESULT, 

AND  RANGE 

TIME 

FIRING 

SIDE 

RESULT 

RANGE 

06:45:00 

Red 

Near  Miss 

14:30 

07:03:00 

Blue 

Hit 

1860 

07:04:00 

Blue 

Hit 

1781 

Near  Miss 

1612 

Red 

Near  Miss 

2263 

Hit 

1856 

07:05:00 

Blue 

Near  Miss 

1563 

Near  Miss 

1836 

Kill 

1132 

Hit 

1894 

Red 

Near  Miss 

1900 

Hit 

1918 
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ROUNDS  OVER  TIME  BY  WEAPON 


Time 


<F>  to  chanoe  scale,  <F>  to  print, 
<ESC>  to  exit. 


Figure  5.  Example  of  a  UPAS  graph  showing  volume  of  flros  as 
function  of  time  and  weapon  type. 


The  Need  to  Expand  UPAS  Capabilities 


SIMNET  is  the  first  example  of  what  is  now  known  more 
broadly  as  Distributed  Interactive  Simulation  (Dis) .  The 
importance  of  unit  collective  performance  measurement  in  the  DIS 
environment  is  illustrated  by  the  inclusion  of  a  Performance 
Measures  Subgroup  in  an  ongoing  series  of  workshops  sponsored  by 
DARPA  and  the  U.S.  Army  Project  Manager  for  Training  Devices 
(PM-TRADE)  to  develop  joint  service  standards  for  DIS. 

In  the  Fourth  Workshop  for  Interoperability  of  Defense 
Simulations,  Meliza  (1991)  described  four  requirements  for  a  unit 
performance  feedback  or  measurement  system  in  the  DIS 
environment.  First,  the  system  must  be  capable  of  integrating 
data  from  multiple  sources.  Second,  the  system  must  help  a 
trainer  to  identify  and  illustrate  key  exercise  events  quickly. 
Third,  the  system  must  be  flexible  to  allow  for  differences  in 
information  requirements  across  exercises  as  a  function  of  such 
variables  as  the  echelon  being  trained,  the  specific  collective 
tasks  to  be  trained,  and  lessons  learned  about  which  types  of 
information  displays  are  most  useful  to  trainers.  Fourth,  the 
system  must  provide  decentralized  data  analysis  to  support 
multiple  AARs.  Each  of  these  requirements  is  described  in  detail 
below  with  a  critique  of  the  capability  of  the  prototype  UPAS  to 
meet  the  requirement. 

Integration  of  Data  Sources 

It  is  necessary  to  integrate  network  data  with  non-network 
data  to  provide  a  complete  description  of  unit  performance. 
Information  about  the  specific  mission,  enemy,  friendly  troops, 
terrain,  and  time  (METT-T)  situation  under  which  an  exercise  is 
performed  is  needed  to  interpret  the  casualty  and  position 
location  data  collected  during  a  simulated  engagement  (Kerrins, 
Atwood,  and  Root, 1990;  Hiller,  1987). 

The  need  to  integrate  network  data  with  other  data  sources 
can  be  demonstrated  by  examining  performance  standards  from  the 
Army  Training  and  Evaluation  Program  (ARTEP)  Mission  Training 
Plan  (MTP)  document  for  Armor  Platoons  (Department  of  the  Array, 
1988a) .  The  frequency  with  which  various  data  sources  and 
combinations  of  data  sources  are  used  in  applying  Armor  Platoon 
MTP  standards  are  shown  in  Table  4.  This  table  provides  a 
summary  of  the  categorization  of  those  data  sources  which  account 
for  at  least  two  percent  of  the  standards  from  the  armor  platoon 
MTP  document.  These  data  were  prepared  using  the  Simulation 
Networking/Training  Requirements  database  (Meliza,  in 
preparation) .  They  include  only  those  standards  supported  by 
SIMNET  (Burnside,  1990) .  Roughly  36%  of  the  standards  require 
integration  of  network  data  with  another  data  source,  while  only 
10%  of  the  standards  can  be  applied  using  network  data  alone. 


13 


TABLR  4.  DATA  SOURCES  USED  IN  APPLYING  ARMOR  PLATOON  MISSION 
TRAINING  PLAN  STANDARDS  SUPPORTED  BY  SIMNET. 


DATA  SOURCES  PERCENTAGE  OF 


STANDARDS  USING  DATA 
SOURCE 

Observations  16% 
Network  10% 
Network  +  Communications  10% 
Network  +  Terrain  9% 
Communications  +  Observation  8% 
Communications  +  Observation  +  Planning  7% 
Observation  +  Planning  7% 
communications  6% 
Network  +  Planning  5% 
Network  +  Planning  +  Terrain  5% 
Network  +  Observation  3% 
Network  +  Communications  +  Planning  2% 
Network  +  Communioations  +  Terrain  2% 


It  is  important  to  note  that  the  "observations"  data  source 
referred  to  in  Table  4  moans  direct  observation  of  the  behavior 
of  individual  soldiers.  Such  observations  can  be  made  by  a 
trainer  in  the  SIMNET  environment  only  when  soldiers  are  outside 
of  the  simulators.  There  have  been  discussions  of  using 
videotaping  or  other  means  to  monitor  the  behavioral  events 
inside  a  simulator,  but  this  approach  appears  to  be  too  costly  to 
implement  on  a  regular  basis.  Therefore,  the  memory  of  exercise 
participants  is  the  major  source  of  information  about  what  would 
otherwise  be  events  observed  by  a  trainer.  Information  about 
these  events  will  tend  to  surface  during  AARs  to  the  extent  that 
they  are  relevant  to  key  exorcise  events. 

The  UPAS  is  concerned  primarily  with  applying  measures  of 
performance  that  make  use  of  the  data  collected  from  the  network. 
Therefore,  we  are  interested  particularly  in  the  subset  of 
standards  that  can  be  trained  in  SIMNET  and  can  be  assessed  using 
network  data  alone  or  network  data  in  combination  with  planning, 
communications,  and  terrain  data.  Focusing  on  these  standards, 
we  identified  twelve  categories  of  standards  that  appear  to  cover 
virtually  all  of  the  MTP  standards  that  meet  those  criteria.  The 
categories  define,  at  a  broad  level,  the  aspects  of  collective 
performance  that  need  to  be  addressed  by  the  UPAS .  These 
categories  are  listed  below  with  more  specific  examples  of  the 
aspects  of  performance  to  be  addressed  within  each  category  and  a 
list  of  the  range  of  data  sources  that  support  the  application  of 
the  standards  in  each  category. 
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friendly  firing  events. 
These  standards  are  used  to  assess  whether  a  unit  takes 
appropriate  movement  actions  when  fired  upon.  The  initial  and 
subsequent  movement  responses  to  enemy  fires  are  a  function  of 
the  overall  METT-T  situation.  This  category  of  standards  also 
addresses  the  coordination  of  friendly  movement  with  friendly 
covering  fires  across  a  wide  range  of  tactical  situations. 

Data  Requirements:  terrain  features;  vehicle  location  (eriemy  and 
friendly);  time,  volume,  and  results  of  enemy  and  friendly  firing 
events;  time  and  location  of  enemy  indirect  fire  missions. 

Friendly  firing  events  cued  bv  enemy  firing  events.  The 
standards  within  this  category  assess  how  well  a  unit  controls 
its  volume  and  distribution  of  fires  in  response  to  enemy  firing 
events.  Do  units  promptly  return  fire?  Do  units  adjust  volume 
and  distribution  of  fires  in  response  to  changes  in  volume  of 
enemy  fires? 

Data  Requirements:  time,  volume,  and  results  of  enemy  and 
friendly  firing  events;  friendly  gun  tube  orientation;  vehicle 
location  (enemy  and  friendly) . 

Compliance  of  movement  with,  control  measures.  These 
standards  are  used  to  aaseas  how  well  a  unit's  movement 
techniques,  routes,  and  movement  rates  match  the  unit's 
operations  order.  Do  units  initiate  movement  and  cross  control 
measures  as  designated  in  the  operations  order?  Do  units  employ 
appropriate  movement  techniques  or  formations  as  a  function  of 
control  measures  das.ignated  in  the  unit  operations  order? 

Data  Requirements:  Location,  purpose,  and  time  data  relevant  to 
each  control  measure;  position  location  data. 

Appropriateness  of  movement  techniques  as  a.  function  of  the 
MFTT~T  situation.  These  standards  are  used  to  assess  whether  the 
movement  techniques  employed  by  the  unit  are  appropriate  to  the 
METT-T  situation  throughout  the  exercise. 

Data  Requirements:  terrain  data;  vehicle  location  (enemy  and 
friendly) ;  tactical  scenario;  enemy  and  friendly  firing  events. 

Use  of  cover  and  concealment  during  movement.  This  category 
of  standards  addresses  the  cover  and  concealment  offered  by  the 
overall  route  of  advance  of  a  unit.  It  also  addresses  the  cover 
and  concealment  afforded  by  short  movements,  such  as  the  route 
from  a  battle  position  to  an  alternate  firing  position. 

Data  Requirements:  terrain  data;  enemy  and  friendly  positions. 
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Orientation  of  weapon  systems  as  a  function  of  the  METT-T 
situation.  This  category  covers  two  subcategories.  The  first 
assesses  whether  the  orientation  of  the  gun  tube  of  each  vehicle 
is  appropriate  given  the  METT-T.  This  second  addresses  the  issue 
of  whether  each  crew  continually  scans  its  assigned  sectors  or 
areas  of  responsibility  as  indicated  by  gun  tube  movement. 

Data  Requirements:  turret  azimuth;  vehicle  orientation; 
enemy  and  friendly  vehicle  positions;  terrain  data. 

Halts  and  cover/concealment.  These  standards  are  used  to 
assess  the  use  of  cover  and  concealment  in  selecting  halt 
positions.  In  the  context  of  offensive  missions,  these  positions 
include  overwatch  positions  selected  prior  to  actual  contact  and 
firing  positions,  in  the  context  of  defensive  missions,  they 
include  primary,  alternate,  and  supplementary  firing  positions. 

Data  Requirements:  enemy  and  friendly  location  data;  terrain 
data. 


Locations  of  friendly  Indirect  fire  relative  to  enemy 
location.  Some  standards  in  this  category  are  concerned  with 
using  indirect  fire  on  known  enemy  locations,  while  others  arc 
concerned  with  using  indirect  fire  on  likely  enemy  locations. 

Data  Requirements:  locations  of  enemy  vehicles;  locations  of 
friendly  indirect  fire  missions;  major  terrain  features;  unit 
plans . 

Spatial  relationships  among  moving  vehicles.  This  category 
of  standards  assesses  the  quality  of  movement  techniques  used  by 
a  unit,  the  location  of  the  Platoon  Leader's  and  Platoon 
Sergeant's  vehicles  relative  to  other  vehicles,  and  whether  an 
appropriate  interval  is  maintained  among  vehicles. 

Data  Requirements;  friendly  vehicle  positions;  terrain  data. 

Rate  of  movement.  This  category  of  standard  addresses 
movement  rates  over  both  short  and  longer  periods  of  time. 
Movement  rates  over  very  short  periods  (a  few  seconds)  are 
examined  to  assess  a  unit's  response  to  an  actual  threat 
situation  (i.e.,  moving  quickly  and  continuously  to  a  covered  and 
concealed  position)  or  to  assess  if  vehicles  move  continuously  at 
critical  points  in  time  to  avoid  blocking  the  movement  of  other 
vehicles  (such  as  when  a  unit  is  moving  into  an  assembly  area  or 
shifting  from  one  formation  to  another) .  Observing  movement 
rates  over  longer  periods  of  time  makes  it  possible  to  assess 
whether  a  unit's  momentum  is  reduced  by  unnecessary  halts  or 
overly  cautious  movement. 

Data  Requirements;  friendly  positions;  terrain  data;  firing 
events;  odometer  readings. 
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Reporting  of  locations  in  terms  of  control  measures .  These 
standards  are  concerned  with  assessing  whether  a  unit  reports  to 
a  higher  headquarters  when  it  reaches  key  locations  or  takes  a 
required  action  at  a  key  location. 

Data  Requirements;  locations  of  friendly  vehicles;  locations  of 
control  measures;  tactical  communications. 

Reporting  of  enemy  contact  and  firing  events.  This  category 
is  concerned  with  assessing  whether  a  unit  reports  initial 
contact,  casualties  inflicted,  and  casualties  sustained. 

Data  Requirements!  tactical  communications;  firing  events. 

We  examined  the  initial  UPAS  system  and  found  that  it 
contains  the  information  needed  to  address  only  one  of  the 
categories  entirely,  friendly  fires  cued  by  enemy  firing  events. 
The  other  eleven  categories  are  only  partially  covered.  Seven  of 
these  categories  could  be  raised  to  fully  covered  with  the 
integration  of  terrain  data  (friendly  movement  cued  by  firing 
events,  use  of  cover  and  concealment  when  moving,  rate  of 
movement,  orientation  of  weapon  systems  as  a  function  of  METT-T, 
spatial  relationships  among  moving  vehicles,  use  of  indirect  fire 
against  the  enemy,  and  use  of  cover  and  concealment  in  halt 
positions) .  The  addition  of  communications  data  would  raise  the 
eighth  category  to  fully  covered  (reporting  of  enemy  contact  and 
firing  events) ,  and  the  addition  of  control  measures  would  add 
the  ninth  category  (compliance  of  movement  with  control 
measures) .  The  addition  of  both  tactical  communications  and 
control  measures  would  add  a  tenth  category  (reporting  of 
position  information) .  The  final  category  could  be  addressed  by 
adding  terrain  data  and  tactical  planning  data  (appropriateness 
of  movement  techniques  as  a  function  of  METT-T) . 

In  summary,  the  prototype  UPAS  needs  to  be  enhanced  to 
support  the  integration  of  network  data  with  other  data  sources. 
Without  this  integration,  at  platoon  level,  only  10%  of  the 
standards  supported  by  SIMNET  are  addressed  effectively  by  the 
UPAS. 

Rapid  Identification  and  Illustration  of  Key  Exercise  Events 

A  performance  measurement  or  feedback  system  must  support 
the  rapid  analysis  and  interpretation  of  data.  Unlike  a  field 
training  exercise,  there  are  few  post-mission  tasks  after  a 
SIMNET  exercise  to  keep  a  unit  occupied  while  a  trainer  analyzes 
unit  performance  in  preparation  for  providing  feedback.  It  is 
critical  that  UPAS  support  the  preparation  of  timely  AARs. 
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The  prototype  UPAS  cannot  meet  this  criterion  fully,  because 
it  fails  to  integrate  network  data  with  other  data  sources. 
However,  the  use  of  the  menu  of  tables  and  graphs  within  the 
UPAS,  combined  with  information  available  in  the  Plan  View, 
does  address  the  10%  of  standards  supported  by  SIMNET  relying 
entirely  on  automated  data. 

We  also  considered  the  amount  of  time  required  to  identify 
and  illustrate  key  exercise  events  using  the  UPAS,  resulting  in 
the  discovery  of  additional  requirements  for  improving  the  UPAS. 
The  Plan  View  is  not  efficient  in  terms  of  its  ability  to  move 
quickly  from  one  point  in  the  battle  to  another.  The  system 
works  by  reading  every  data  packet  in  the  sequence  in  which  the 
packets  were  collected.  Although  the  Plan  View  has  a  nominal 
search  function  and  the  capability  to  move  through  the  sequence 
of  events  from  the  Mission  Event  List,  it  must  cycle  through 
every  interpolated  data  packet  in  moving  from  one  point  in  time 
to  another.  As  a  result,  there  is  no  time  savings  gained  by 
using  the  "search  function"  on  the  Plan  view. 

The  speed  of  the  Plan  View  replay  is  not  selectable  by  the 
user.  Replay  speed  is  a  function  of  the  operating  speed  of  the 
host  PC  and  the  number  of  data  packets  being  generated  per  unit 
of  time.  When  there  is  little  action  during  an  exercise,  the 
replay  is  conducted  at  high  speeds  (roughly  4  x  real  time  with  a 
Zenith  286  and  roughly  15  x  real  time  with  a  Zenith  386  operating 
at  25  megahertz) .  At  points  in  an  exercise  when  the  level  of 
movement  and  firing  activities  are  high,  generating  large  numbers 
of  data  packets,  the  replay  slows  down  to  speeds  slower  than  real 
time. 


In  summary,  two  types  of  improvements  are  required  in  the 
UPAS  if  it  is  to  support  the  identification  and  illustration  of 
key  exerf  ise  events  in  a  comprehensive  and  timely  manner.  First, 
other  data  sources  must  be  integrated  with  network  data,  second, 
the  UPAS  Plan  View  must  be  enJianced  to  allow  the  user  to  move 
quickly  from  one  point  in  the  exercise  to  another. 

Flexibility 

The  UPAS  should  be  flexible  enough  to  allow  the  outputs  of 
the  system  to  be  modified  easily  without  major  reprogramming  of 
software.  The  military  still  has  very  limited  experience 
applying  DIS  to  training,  and  little  data  are  available  that  can 
be  used  to  justify  detailed  formats  for  measurement  outputs  in 
terms  of  their  utility  to  users.  The  UPAS  should  be  viewed  as  a 
tool  that  will  allow  us  to  try  out  different  output  formats 
during  the  initial  fielding  of  a  DIS  application  and  make  prompt 
changes  in  response  to  these  tryouts.  Rigid,  formal  programming 
of  the  entire  performance  measurement  system  will  not  provide  us 
with  the  flexibility  to  refine  the  system  promptly  at  a 
reasonable  cost. 
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The  prototype  UPAS  offers  flexibility  by  giving  users  the 
capability  to  modify  graphs  and  tables  based  on  data  from  the 
SIMNET/NTC  database.  To  accomplish  such  modifications,  the  user 
needs  to  know  SQL  and  the  UPAS  procedures  for  defining  graphs  and 
tables. 

The  prototype  UPAS  also  provides  the  user  with  the 
flexibility  to  change  the  scale  of  the  Plan  View  to  focus  on  the 
action  in  a  small  part  of  the  battlefield  or  examine  the  battle 
from  a  broad  perspective.  However,  the  UPAS  provides  the  user 
with  little  or  no  capability  to  control  the  speed  of  the  replay 
or  the  point  in  time  in  the  battle  to  be  replayed. 

In  summary,  the  prototype  UPAS  offers  the  user  the  ability 
to  modify  AAR  aids  to  meet  diverse  information  needs.  The 
prototype  UPAS  is  also  a  tool  that  can  be  used  to  try  out  the 
effectiveness  of  specific  tables  and  graphs,  and  it  can  be  used 
to  modify  these  tables  and  graphs  in  response  to  lessons  learned 
about  the  adequacy  of  feedback. 


Decentralized  Data  Analysis  to  Support  Multiple  After  Action 
Reviews 


The  performance  measurement  system  must  provide 
decentralized  data  analysis  to  support  multiple  After  Action 
Reviews  (AARs) ,  with  each  AAR  tailored  to  meet  the  information 
needs  of  a  distinct  population.  For  example,  after  a  company 
team  exercise,  at  least  four  separate  AARs  need  to  be  supported; 
each  of  the  armor  and  infantry  platoons  should  be  provided  with 
feedback  about  how  well  their  crews  worked  together  in  performing 
their  platoon  tasks,  and  the  company  team  should  receive  feedback 
about  how  well  its  platoons  worked  together.  As  the  scope  of  an 
exercise  is  increased  to  include  highc  echelons  and  more  types 
of  units,  the  number  of  groups  requirin  tailored  feedback 
increases. 

Since  UPAS  is  a  low-cost  PC-based  system,  multiple  PCs  with 
UPAS  software  may  be  used  to  collect  data  from  the  same  exercise. 
Each  UPAS  could  then  be  used  to  identify  and  illustrate  key 
events  from  the  perspective  of  a  different  unit.  For  example, 
one  system  could  be  used  to  support  a  company-level  AAR,  while 
three  other  systems  might  be  used  to  prepare  A7R  materials  for 
each  of  the  three  platoons  in  the  exercise,  re.-.pectively .  It  is 
important  to  note  that  a  single  PC  could  be  used  to  support 
all  four  AARs  if  necessary.  The  use  of  multipj.e  PCs  is 
advantageous,  because  it  reduces  the  amount  of  time  required  to 
prepare  AAR  aids. 
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Enhanced  After  Action  Review  Aids 

We  designed  concepts  for  improving  the  Plan  view  and 
creating  new  types  of  AAR  aid  formats  to  support  AARs  more 
effectively.  Our  major  goals  in  developing  these  concepts  were 
to  facilitate  the  integration  of  network  data  with  other  data 
sources  and  expedite  the  preparation  of  AAR  aids.  The  University 
of  Central  Florida  Institute  for  Simulation  and  Training  (1ST) 
implemented  these  concepts  through  software  development. 

In  developing  AAR  aid  concepts,  a  heavy  emphasis  was  placed 
upon  the  use  of  figures  for  two  reasons.  First,  figures  were 
believed  to  be  a  useful  vehicle  for  integrating  network  and  non¬ 
network  data.  Second,  figures  were  believed  to  be  a  good  vehicle 
for  summarizing  unit  performance  in  a  manner  that  can  be 
interpreted  quickly  by  experienced  trainers. 

In  addition  to  providing  ideas  for  improving  the  Plan  View 
Display,  we  developed  concepts  for  three  new  types  of  UPAS  AAR 
aids;  Battle  Flow  Charts,  Battle  Snapshots,  and  Exercise 
Timelines.  The  design  concept  for  each  type  of  aid  is  described 
below,  and  each  description  includes  a  listing  of  the  categories 
of  performance  measures  that  might  be  applied  with  each  aid. 

Improved  Plan  View  Display 

Major  terrain  features  (highways,  unimproved  roads, 
buildings,  bodies  of  water,  treelines,  and  clumps  of  trees) , 
other  than  contour  lines,  were  added  to  the  Plan  View.  In 
addition,  unit  control  measures  from  the  unit's  operations  order 
were  added  to  the  Plan  View.  An  example  of  the  improved  Plan 
View  is  shown  in  Figure  6. 

In  addition  to  integrating  network  and  non-network  data,  the 
improved  Plan  view  Display  also  includes  two  new  features  that 
make  it  easier  to  use.  First,  a  rapid  search  capability  has  been 
added  to  make  it  possible  to  move  forward  and  backward  from  one 
point  in  an  exercise  to  another  quickly.  This  feature  applies  to 
both  the  search  function  and  Mi.ssion  Event  List  function  of  UPAS. 
Second,  the  capability  to  magnify  the  battlefield  has  been 
increased  by  making  it  possible  to  display  a  section  of  the 
battlefield  as  small  as  200  meters  by  200  meters. 

The  categories  of  performance  measures  that  can  be  applied 
using  the  improved  Plan  View  Display  are  friendly  movement  cued 
by  firing  events,  spatial  relationships  among  moving  vehicles, 
friendly  firing  events  cued  by  enemy  firing  events,  weapon  system 
orientation  as  a  function  of  METT-T,  and  rate  of  movement.  Until 
contour  lines  are  added  to  the  Plan  View,  it  will  partially 
address  use  of  cover  and  concealment  during  movement  and  halts, 
and  it  will  partially  addresf'.  the  appropriatenes.s  of  movement 
techniques  as  a  function  of  the  METT-T  situation. 
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PLAN  VIEW 


TIME:  1020 

NEXT  EVENT:  CROSS  LD  EVENT  TIME:  1000 


Figure  6.  improved  Plan  view  Display. 

The  addition  of  control  measures  to  the  Plan  View  makes  it 
possible  to  track  unit  actions  and  rate  of  movement  as  a  function 
of  the  planned  use  of  control  measures.  Because  the  Mission 
Event  List  is  linked  to  the  replay  of  the  exercise  in  the  Plan 
View,  the  user  can  move  from  one  time- tagged  event  to  another  to 
assess  whether  the  unit  is  following  the  rate  of  movement  implied 
in  the  operations  order,  and  it  can  be  used  to  assess  whether  the 
unit  is  taking  any  required  actions  when  it  reaches  a  control 
measure.  The  types  of  actions  at  control  measures  that  can  be 
assessed  include  changing  movement  formations,  changing  the 
orientation  of  weapon  systems,  and  moving  into  a  halt  position. 

In  order  to  display  the  location  of  control  measures,  the 
user  must  first  load  information  about  these  measures  into  the 
SIMNET/NTC  database.  This  information  is  taken  from  map  overlays 
or  other  graphics  and  loaded  using  a  series  of  menus.  The  first 
menu  allows  the  user  to  select  a  control  measure  from  the 
following  options:  Assembly  Areas,  Line  of  Departure,  Phase 
Lines,  Start  Points,  Release  Points,  Check  Points,  and 
Objectives.  The  user  is  then  asked  to  type  in  the  name  of  the 
control  measure  and  the  UTM  coordinates  that  define  its  location. 
Points  are  identified  with  a  single  coordinate,  lines  with  a 
minimum  of  two  coordinates,  and  areas  with  a  minimum  of  three 
coordinates.  After  defining  one  control  measure,  the  user  is 
returned  to  the  original  menu  to  select  another  control  measure. 
Figure  7  illustrates  the  series  of  menus  used  to  load  information 
about  unit  control  measures. 


22 


Figure  7.  Series  of  menus  used  to  load  information  about  unit 
control  measures  into  the  SIMNET/NTC  database. 

Battle  Flow  Chart 

k  Battle  Flow  Chart  is  an  animated  figure  that  traces  the 
movement  of  vehicles  and  units  throughout  the  course  of  a  mission 
or  during  a  significant  segment  of  a  mission.  The  trace  is 
implemented  by  showing  vehicle  locations  at  a  predetermined 
interval,  such  as  every  minute.  Movement  is  displayed  over  a 
grid  map  that  includes  major  terrain  features  and  the  control 
measures  tor  a  particular  mission  (Figure  8),  The  user  can  stop 
at  any  point  during  the  trace  of  the  movement  of  vehicles  and 
create  a  hard  copy  of  the  display. 

The  icons  used  to  represent  vehicle  locations  are  plus  signs 
and  numerals.  Unlike  the  case  with  the  Plan  view  Display,  the 
icons  provide  no  information  about  weapon  system  orientation. 

The  Battle  Flow  allows  the  user  to  specify  the  time  interval  at 
which  vehicle  positions  are  to  be  marked.  This  important  feature 
allows  one  to  adjust  the  position  updates  to  avoid  cluttering  the 
screen  with  data.  As  in  the  case  of  the  Plan  View,  the  Battle 
Flow  also  allows  the  user  to  magnify  portions  of  the  battlefield 
of  particular  interest  up  to  the  point  where  the  entire  display 
is  filled  by  an  area  that  is  only  200  meters  by  200  meters. 
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Events  and  relationships  that  might  be  assessed  with  this 
aid  are  compliance  of  movement  with  control  measures,  movement 
rate,  appropriateness  of  movement  techniques  as  a  function  of  the 
METT-T  situation,  use  of  cover  and  concealment  during  movement, 
and  use  of  cover  and  concealment  when  halted. 


BATTLE  FLOWS 


DATE:  91-8-21  START  TIME:  1650  FINAL  TIME:  1708 

EXERCISE  ID:  001  COMPANY:  A  PLATOON:  1 


Figure  8,  Sample  Battle  Flow  Chart. 


A  Battle  Snapshot  is  another  tool  that  shows  the  position  and 
orientation  of  vehicles  and  weapon  systems  from  a  bird's-eye  view 
(See  Figure  9) .  A  Snapshot  can  be  taken  at  any  time  in  the 
exercise  designated  by  the  user.  Like  the  Plan  View  and  the 
Battle  Flow,  the  Snapshot  allows  the  user  to  magnify  the 
battlefield.  Like  the  Battle  Flow,  the  Snapshot  shows  the 
location  of  vehicles  on  a  grid  map  that  includes  control 
measures.  These  control  measures  are  taken  from  the  SIMNET/ 

NTC  database  in  the  same  manner  as  for  the  Battle  Flow  Chart. 
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Like  the  Plan  View,  vehicles  are  represented  by  oversized 
icons  that  indicate  vehicle  and  weapon  system  orientation. 
Performance  aspects  that  can  be  assessed  with  this  aid  are 
compliance  of  movement  with  control  measures,  spatial 
relationships  among  moving  vehicles,  and  weapon  system 
orientation  as  a  function  of  METT-T. 


BATTLE  SNAPSHOT 


DATE:  91-8-21  TIME:  1658 

EXERCISE  ID:  001  COMPANY:  A  PLATOON:  1 


Figure  9.  Sample  Battle  Snapshot. 


An  Exercise  Timeline  is  a  tool  for  looking  at  temporal 
coordination  of  movement,  control  measures,  and  firing  events. 
The  top  line  of  the  Timeline  in  Figure  10  indicates  time  of  day 
during  the  exercise.  The  second  line  describes  movement  of  the 
platoon  as  a  function  of  time  and  unit  control  measures.  The 
bars  at  the  bottom  of  this  line  indicate  the  time  when  the  first 
and  last  vehicle  of  a  unit  crossed  a  control  measure.  This 
information,  combined  with  time-tagged  events  from  the  unit  plan 
contained  in  the  Mission  Event  List,  provides  a  quick  method  for 
comparing  the  planned  and  actual  rate  of  movement  of  a  unit. 
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SIMNET  EXERCISE  TIMELINE 


DATE:  10/12/91  EXERCISE  ID:  M001  COMPANY:  A  PLATOON:  1 


LEGEND: 


■  TIME  BETWEEN  FIRST  AND  LAST  VEHICLE  CROSSING  THE  CONTROL  MEASURE 
TIME  DURING  WHICH  NO  VEHICLE  MOVED  \  FIRST  FRIENDLY  FIRE  DELIVERED 
□  ARTILLERY  FIRE  NEAR  UNIT  X  ENEMY  VEHICLE  DESTROYED 

{  FIRST  ENEMY  FIRE  RECEIVED  O  FRIENDLY  VEHICLE  DESTROYED 


Figure  10.  Sample  Exercise  Timeline. 

The  second  line  of  the  Exercise  Timeline  also  indicates  the 
beginning  and  ending  of  periods  in  time  when  the  entire  platoon 
was  halted.  Information  about  halts  may  be  used  in  a  variety  of 
ways.  First,  they  can  be  used  to  compare  the  halts  employed  with 
the  planned  halts.  Second,  they  can  be  used  to  identify  points 
in  time  when  the  movement  of  the  unit  was  bogged  down  by 
unnecessary  halts,  indicating  a  command  and  control  or  land 
navigation  problem.  Third,  these  data  may  be  used  to  examine  the 
appropriateness  of  halts  as  a  function  of  direct  fire  and 
indirect  fire  events  indicated  on  the  third  line  of  the  Timeline. 

The  third  line  of  the  Exercise  Timeline  provides  information 
about  the  time  of  direct  and  indirect  firing  events.  A  small 
square  is  used  to  indicate  when  the  unit  receives  artillery  fire, 
and  an  arrow  pointed  down  indicates  when  the  first  enemy  direct 
fire  was  received  by  the  unit.  A  small  circle  is  used  to 
indicate  when  a  friendly  vehicle  is  destroyed.  An  arrow  pointed 
up  indicates  when  the  unit  first  delivers  fire  on  the  enemy,  and 
a  small  "x”  indicates  when  an  enemy  vehicle  is  de.p*->'oyed. 
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A  fourth  line  is  envisioned  for  the  Exercise  Timeline  to 
provide  information  about  the  time  of  tactical  communications. 

At  a  minimum,  we  would  like  this  line  to  indicate  whether  a  unit 
contacted  a  higher  headquarters  around  the  time  oi  any  major 
tactical  events  indicated  in  lines  2  and  3  of  the  Timeline.  For 
example,  did  the  unit  communicate  with  its  headquarters  after 
receiving  direct  or  Indirect  fires?  Further,  we  would  like  to 
know  the  type  of  radio  message  and  the  content  of  the  message. 

Unfortunately,  because  the  UPAS  is  PC-based,  it  can  only 
perform  one  task  at  a  time.  During  a  SIMNET  exercise,  use  of  the 
UPAS  is  limited  to  collecting  data  from  the  simulation  network, 
and  it  cannot  be  used  to  collect  data  on  tactical  radio 
communications.  However,  an  effort  is  under  way  to  apply  a 
separate  PC  to  the  task  of  monitoring  radio  communications.  The 
output  of  the  radio  monitoring  activities  will  be  loaded  into  the 
SIMNET/NTC  database  at  the  end  of  an  exercise. 
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Concepts  for  Applying  UPAS  AAR  Aids 
to  the  Task  of  Identifying  Key  Exercise  Events 


Two  considerations  are  critical  in  developing  a  plan  for 
applying  AAR  aids  to  perforinance  measurement.  First,  one  must 
consider  which  of  the  categories  of  standards  mentioned  in  a 
previous  section  of  this  report  are  addressed  by  each  type  of  AAR 
aid.  Second,  one  must  consider  how  to  navigate  through  the  AAR 
aids  in  an  efficient  manner. 

Table  5  shows  which  categories  of  standards  are  appropriate 
to  each  AAR  aid  format.  This  table  also  serves  to  illustrate  the 
unique  contributions  made  by  the  various  aids  in  addressing  MTP 
standards,  as  well  as  illustrating  their  overlap.  The  logic 
behind  the  assignments  for  categories  of  standards  which  might  be 
addressed  by  each  type  of  AAR  aid  is  provided  below. 

Friendly  Movement  Cued  bv  Enemy  or  Friendly  Firing  Events 

The  Plan  View  is  the  only  figure  that  displays  movement  of 
both  enemy  and  friendly  vehicles  with  continuous  updates.  The 
Plan  View  also  indicates  vehicle  firing  events  as  a  function  of 
time.  To  the  extent  that  the  Plan  View  displays  information 
about  terrain  features,  it  would  appear  to  support  this  category 
of  standards.  The  Battle  Flow  chart  and  Battle  Snapshot,  on  the 
other  hand,  are  not  expeoted  to  provide  information  on  enemy 
firing  events.  This  gap  in  information,  combined  with  the  lack 
of  continuous  position  updates,  suggests  these  two  aids  would  not 
support  this  category  of  standards.  The  Exercise  Timeline 
partially  supports  this  category  of  standards  by  providing 
information  about  the  temporal  relationships  between  movement  of 
the  unit  as  a  whole  and  firing  events  (enemy  and  friendly) . 
However,  the  Timeline  lacks  information  on  terrain  features, 
volume  of  enemy  fires,  and  movement  of  individual  vehicles. 

Friendly.  Firing  Events  cued  bv  Enemy  Firing  Events 

Tables  and  graphs  describe  time,  volume,  and  effectiveness 
relationships  between  enemy  and  friendly  fires.  The  Plan  View 
provides  this  sane  information  but  in  a  form  that  is  less  concise 
and  precise  than  that  available  with  tables  or  graphs.  Battle 
Flow  Charts  and  Battle  Snapshots  are  simply  not  expected  to 
provide  firing  event  data.  The  Timeline  is  expected  to  contain 
some  information  about  the  temporal  relationships  between  enemy 
and  friendly  firing  events,  but  this  information  does  not  include 
volume  of  fires. 
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TABLE  5.  CATEGORIES  OF  STANDARDS  APPROPRIATE  TO  EACH  TYPE  OF  AAR 
AID 


AAR  AID  FORMAT  | 

CATEGORY  OF  STANDARDS 

TABLES/ 

GRAPHS 

FLOW 

CHART 

SNAPSHOT 

PVD 

TIMELINE 

MOVEMENT  AND  FIRING  EVENTS 

• 

• 

FRIENDLY  AND  ENEMY  FIRES 

• 

• 

MOVEMENT  AND  CONTROL  MEASURES 

• 

• 

• 

• 

MOVEMENT  TECHNIQUE  AND  METT^T 

• 

• 

• 

• 

MOVEMENT  AND  COVER/CONCEALMENT 

• 

• 

• 

WEAPON  ORIENTATION 

• 

• 

HALTS  AND  COVER/CONCEALMENT 

• 

• 

• 

• 

LOCATIONS  OP  FRIENDLY  INDIRECT 

FIRE  AND  ENEMY  POSITIONS 

• 

SPATIAL  RELATIONSHIPS  AMONG 

MOVING  VEHICLES 

• 

• 

• 

RATE  OF  MOVEMENT 

• 

• 

LOCATION,  CONTROL  MEASURES, 

AND  COMMUNICATIONS 

FIRING  EVENTS  AND  COMMUNICATIONS 

*  R«quir«*  addition  of  oommuniontion#  data  to  tho  Bxaroise 
Tlmallno 


compliance  of  Movement  With  contra.!  MgaBUr.ea 

Every  AAR  aid  can  be  used  to  at  least  estimate  the  distance 
between  vehicles  and  control  measures.  The  Exercise  Timeline 
would  indicate  when  a  unit  crossed  or  reached  a  control  measure 
(based  on  a  precise  measurement  of  the  distance  between  vehicles 
and  the  control  measure) . 
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Tables,  graphs,  and  Exercise  Timelines  would  not  be  useful 
for  this  category  of  standards,  because  they  provide  no 
information  about  movement  techniques.  The  other  three  types  of 
aids  are  expected  to  display  vehicle  positions  in  a  manner  that 
would  allow  a  trainer  or  researcher  to  identify  the  movement 
technique  being  employed.  Because  these  aids  also  contain 
information  about  the  METT-T  situation  (such  as  the  possibility 
of  imminent  enemy  contact,  terrain  f'atures),  they  can  be  used  to 
assess  the  quality  and  appropriateness  of  movement  techniques. 

Use  of  Cover  and  Concealment  Purina  Movement 

To  apply  standards  in  this  category,  one  must  have 
information  about  the  location  of  friendly  elements  during 
movement,  terrain  features,  and  the  location  of  enemy  elements. 
The  Plan  View,  Battle  Flow,  and  Battle  Snapshot  are  all  expected 
to  display  friendly  and  enemy  positions  relative  to  major  terrain 
features. 

Weapon  System  Orientation 

The  Plan  View  and  Battle  Snapshot  are  expected  to  display  the 
orientation  of  gun  tubes,  and  the  location  of  enemy  positions. 
Further,  since  gun  tube  orientation  is  often  a  function  of 
movement  techniques,  it  is  important  to  note  that  both  of  these 
displays  are  expected  to  make  it  possible  to  identify  the 
movement  technique  being  employed  by  a  unit  (as  described  above) . 
Only  the  Plan  View  and  Battle  Snapshot  are  expected  to  display 
weapon  system  (turret  or  gun  tube)  orientation.  Of  these  two 
aids,  only  the  Plan  View  is  capable  of  the  animation  necessary  to 
assess  turret  movement.  However,  a  sequence  of  Battle  Snapshots 
might  be  used  to  assess  whether  weapon  system  orientation  is 
changing  over  short  periods  of  time. 

Use  of  Cover  and  Concealment  in  Halt  Positions 

The  Plan  View,  Battle  Snapshot,  and  Battle  Flow  Chart 
provide  information  about  terrain  features,  threat  positions,  and 
friendly  positions  that  can  be  used  in  applying  this  category  of 
standards.  With  the  Plan  View  and  the  Battle  Flow  it  should  be 
possible  to  decide  whether  a  unit  is  in  a  halt  position,  but  with 
the  Battle  snapshot,  the  user  would  have  to  know  when  a  unit  was 
halted  before  creating  a  Snapshot. 
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Tables  and  graphs  are  likely  to  be  the  best  data  source  for 
applying  this  category  of  standards.  Tables  are  expected  to 
provide  information  about  the  time,  location,  and  results  of 
indirect  fire  missions,  as  well  as  information  about  the  exact 
location  of  enemy  forces  during  friendly  indirect  fire  missions. 
Addressing  locations  of  enemy  forces  and  friendly  indirect  fire 
missions  in  an  Exercise  Timeline  would  be  entirely  outside  the 
concept  of  this  aid.  The  Plan  View,  Battle  Flow,  and  Battle 
Snapshot  are  all  expected  to  display  the  location  of  enemy 
vehicles  and  major  terrain  features,  but  none  of  these  aids 
display  the  location  of  friendly  indirect  fire  missions.  Given 
that  tables  and  graphs  are  expected  to  address  this  category  of 
standards  adequately,  and  given  the  possibility  that  adding 
indirect  fire  locations  would  create  a  screen  clutter  problem  for 
the  three  types  of  bird's-eye  view  of  the  battlefield,  trying  to 
use  these  three  aids  to  address  this  category  of  standards 
appears  to  be  unwarranted. 

Spatial  Relationships  Among  Moving  Vehicles 

The  Plan  View,  Battle  Snapshot,  and  Battle  Flow  are  all 
expected  to  display  the  location  of  vehicles  in  adequate  detail 
to  allow  trainers  and  researchers  to  decide  what  kind  of 
formation  a  unit  is  employing  at  a  particular  time.  Tab]es  and 
graphs  might  conceivably  be  used  to  assess  formations  in  terms  of 
geometric  or  trigonometric  relationships,  but  this  would  appear 
to  be  a  needlessly  complex  way  of  supporting  a  decision  in 
comparison  with  a  bird's-eye  view. 

Rate  of  Movement 

The  majority  of  standards  in  this  category  require  that  a 
unit  move  continuously  without  stopping.  For  selected  standards 
there  is  also  the  requirement  that  vehicles  move  quickly,  of  the 
three  figures,  only  the  Plan  View  is  capable  of  continuously 
monitoring  movement.  The  fact  that  position  data  are  updated 
within  the  Plan  View  on  a  continuous  basis  offers  the  potential 
to  look  at  disruptions  of  movement  in  individual  vehicles  during 
tiny  movement  segments,  such  as  the  transition  from  one  formation 
to  another.  The  Exercise  Timeline  is  expected  to  identify 
intervals  during  which  all  vehicles  in  a  unit  are  halted  so  that 
this  aid  should  be  a  good  source  of  information  about  when  the 
movement  of  the  unit  as  a  whole  is  stopped  or  delayed  seriously. 

Reporting  of  Locations  in  Terms  of  Control  Measures 

The  Exercise  Timeline  is  the  most  appropriate  aid  to  address 
this  category  of  standards.  The  Timeline  is  the  only  aid 
expected  to  address  tactical  communications.  Further,  the 
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Timeline  is  expected  to  provide  the  most  precise  information 
about  when  a  unit  crossed  or  reached  a  particular  control  measure 
(as  described  in  the  section  above  on  "compliance  of  movement 
with  control  measures"). 

Reporting  of  Enemy  Contact  and  Firing  Events 

The  Exercise  Timeline  is  also  the  most  appropriate  aid  to 
address  this  category  of  standards,  because  it  is  the  only  aid 
expected  to  address  tactical  communications.  Further,  the 
Timeline  is  expected  to  show  the  temporal  relationship  between 
tactical  communications  and  many  of  the  firing  and  contact  events 
calling  for  communications  (e.g.,  first  enemy  fite  received, 
first  friendly  fire  delivered,  destruction  of  enemy  or  friendly 
vehicles) . 

In  addition  to  considering  the  aspects  of  unit  performance 
that  can  be  addressed  with  each  aid,  it  is  important  to  consider 
how  each  type  of  aid  can  be  used  most  efficiently.  Efficient  use 
of  the  Plan  View  and  Battle  Snapshot  requires  that  a  trainer  know 
the  times  when  key  events  occurred  during  and  exercise.  The 
times  listed  for  planned  exercise  events  in  the  Mission  Event 
List  provides  one  means  of  identifying  times  in  the  exercise  that 
are  potentially  key  to  the  outcome.  A  trainer  can  identify  other 
key  times  using  the  overviews  of  the  exercise  provided  by  the 
Exercise  Timeline  and  the  Battle  Flow.  For  example,  the  Timeline 
would  indicate  when  a  unit  first  was  fired  upon  by  the  threat 
force,  and  it  would  indicate  each  time  when  the  unit  received 
indirect  fire,  as  well  as  providing  information  about  the  unit's 
rate  of  movement  as  a  function  of  control  measures,  Therefore, 
the  most  efficient  strategy  for  using  the  UPAS  AAR  aids  calls  for 
using  the  Exercise  Timeline  and  Battle  Flow  Chart  first  to 
provide  an  overview  of  the  exercise  and  to  identify  exercise  key 
events. 
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Summary  and  Future  Considerations 


The  Improved  UPAS 

Figure  11  provides  an  overview  of  the  improved  UPAS, 
indicating  new  system  components  with  asterisks.  The  Plan  View, 
Battle  Flow,  and  Battle  Snapshot  use  the  data  packets  collected 
from  the  network,  while  the  Exercise  Timeline  used  the  data  that 
have  been  converted  and  loaded  into  the  SIMNET/NTC  relational 
database. 

The  improved  UPAS  requires  the  user  to  input  data  on  the 
locations  of  unit  control  measures  from  the  unit's  operations 
order.  These  data  are  loaded  into  the  SIMNET/NTC  relational 
database  in  a  manner  transparent  to  the  user.  The  UPAS  then 
displays  these  measures  in  the  Plan  View,  Battle  Flow,  and  Battle 
snapshot  to  help  integrate  unit  planning  data  with  network  data. 
In  addition,  the  program  for  the  Exercise  Timeline  uses  control 
measure  data  and  network  data  to  automatically  identify  the  point 
in  time  when  a  unit  crosses  each  control  measure. 


Figure  11.  Overview  of  major  components  of  the  improved  UPAS. 
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The  SIMNET  terrain  database  has  been  added  to  the  UPAS  to 
support  the  display  of  major  terrain  features  in  the  Plan  View, 
Battle  Flow,  and  Battle  Snapshot.  This  addition  supports  the 
integration  of  terrain  data  with  network  data. 

The  integration  of  network  data  with  unit  planning  data  and 
terrain  data  in  the  UPAS  triples  the  number  of  Armor  Platoon  MTP 
standards  that  can  be  applied  during  SIMNET  exercises.  The 
planned  addition  of  communications  data  to  the  Exercise  Timeline 
will  result  in  a  quadrupling  of  the  standards  addressed  by  the 
UPAS,  in  comparison  with  the  prototype  UPAS.  Further,  the 
standards  addressed  by  the  UPAS  are  believed  to  be  especially 
important,  because  they  tend  to  be  concerned  with  assessing  the 
outcome  of  the  movement,  shooting,  and  communication  tasks  that 
are  critical  to  armor  and  infantry  platoon  missions. 

The  UPAS  AAR  aids  require  user  testing.  One  goal  of  this 
testing  is  to  decide  whether  these  aids  are  capable  of  supporting 
the  application  of  performance  standards  assigned  to  each  aid. 

For  example,  we  estimate  that  the  Battle  Flow  Chart  will  trace 
the  movement  of  a  unit  in  adequate  detail  to  allow  the  UPAS  user 
to  identify  the  specific  movement  technique  used  by  a  unit  at 
particular  points  within  a  mission.  During  user  testing  we  may 
find  that  the  Battle  Flow  is  inadequate  to  meet  this  need. 

Application  to  Training  Above  Platoon  bevel 

The  concepts  for  AAR  aids  described  in  this  report  also 
apply  to  training  at  company  team  (Department  of  the  Army,  1988b) 
and  battalion  task  force  (Department  of  the  Army,  1988c)  levels. 
However,  the  categories  of  standards  and  the  types  of  information 
to  be  included  in  each  type  of  aid  are  expected  to  differ  from 
platoon  level,  because  the  topics  addressed  by  training  differ 
across  echelons.  For  example,  platoon  level  training  is 
concerned  with  training  crews  to  work  together  as  a  team,  while 
company  team  level  exercises  emphasize  training  platoons  to  work 
together  as  a  team.  The  analytical  procedures  used  to  define  the 
functional  requirements  at  platoon  level  should  be  employed  at 
company  level  and  above  to  define  the  AAR  needs  at  these 
echelons.  The  Burnside  (1990)  report  provides  the  beginning  of 
such  an  analysis  by  identifying  company  team  and  battalion  task 
force  MTP  standards  that  are  supported  by  SIMNET. 

Application  to  Future  Generations  of  Networked  Simulators 

In  the  near  future,  networked  simulators  are  expected  to 
become  a  major  tool  in  the  conduct  of  training  for  Army  units, 
and  they  are  expected  to  incorporate  more  combined  arms  elements. 
The  next  generation  SIMNET,  the  Close  Combat  Tactical  Trainer 
(CCTT) ,  will  make  it  possible  for  armor  and  infantry  units  to 
interact  with  engineer,  aviation,  artillery,  air  defense,  and 
logistical  units  on  a  simulated  battlefield  (U.S.  Army  Armor 
School,  1990). 
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At  the  start  of  this  effort,  an  expanded  UPAS  was  viewed  as 
a  tool  that  could  be  used  for  training  research  to  develop 
information  pertinent  to  the  future  SIMNET,  CCTT,  as  well  as 
providing  information  about  how  to  best  employ  the  current 
SIMNET.  Due  to  recent  events,  UPAS  has  the  potential  to  serve  as 
a  tool  for  developing  feedback  systems  and  training  strategies 
for  a  wide  variety  of  applications  far  into  the  future. 

All  future  applications  of  DIS,  including  CCTT,  will  be 
interoperable  (McDonald,  Pinon,  Glasgow,  and  Danisas,  1990) ,  and 
the  functional  specifications  for  CCTT  require  that  CCTT  be 
interoperablo  with  SIMNET  (U.S.  Army  Armor  School,  1990). 
Therefore,  a  translator  between  the  standard  protocol  and  the 
SIMNET  protocol  will  be  developed  that  will  link  UPAS  to  all 
future  DIS  applications. 

By  basing  the  design  of  the  UPAS  relational  database  on  the 
design  of  the  NTC  database,  the  UPAS  is  already  prepared  to 
address  combined  arms  elements  that  are  not  included  in  SIMNET 
but  will  be  added  in  future  DIS  applications.  Further,  in  the 
Fourth  Interservice  DIS  Meeting  a  list  of  automated  data 
collection  requirements  was  developed  that  cuts  across  the 
services,  and  this  list  overlaps  substantially  with  the  list  of 
data  elements  in  the  UPAS  database.  The  unique  requirements  not 
addressed  by  UPAS  are  in  the  areas  of  "switchology"  and 
electromagnetic  emitters. 

Finally,  the  lessons  learned  from  UPAS  development  are  being 
used  as  a  basis  for  the  preparation  of  interservice  standards  for 
the  interoperability  of  defense  simulations.  More  specifically, 
these  lessons  were  used  in  preparing  a  draft  standard  for  a  unit 
performance  feedback  system  at  the  Fifth  Workshop  on  Standards 
for  the  Interoperability  of  Defense  Simulations. 

Application  to  SAFOR  Performance  Measurement 

Current  interest  in  SAFOR  behavior  includes  describing  the 
expected  behavior  and  deciding  how  to  measure  this  behavior  to 
insure  that  expectations  have  been  met  (Mullally,  Petty,  and 
Smith,  1991) .  SAFOR  modeling  includes  the  behavior  of  enemy  and 
friendly  units.  Both  types  of  SAFOR  have  the  same  job,  cuing  and 
reinforcing  the  actions  of  the  unit  to  be  trained.  Descriptions 
of  the  expected  actions  of  SAFOR  must  be  extracted  from  MTP 
documents  through  careful  analysis  of  task  conditions,  task 
standards,  subtasks,  and  subtask  standards. 

SAFOR  measurement  techniques  are  needed  to  assess  the 
effectiveness  of  tools  being  developed  to  control  behavior  of  the 
SAFOR,  and  the  UPAS  can  meet  this  need.  Unlike  measuring  the 
performance  of  actual  units,  SAFOR  measurement  involves  only  four 
sources  of  data;  network  data,  terrain  data,  tactical 
communications  (simulated  communications  from  friendly  units) , 
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and  unit  plans  (when  SAFOR  are  used  to  play  the  role  of  friendly 
forces) .  Three  of  these  four  data  sources  are  addressed  by  the 
current  UPAS.  The  addition  of  tactical  communications  to  the 
UPAS  in  the  near  term  will  allow  it  to  address  all  aspects  of 
SAFOR  performance. 

Application  to  Performance  Measurement  systems  for  Embedded 


Embedded  Training  (ET)  is  "training  that  is  provided  by 
capabilities  designed  into  or  added  in  to  operational  systems" 
(Department  of  the  Army,  1987).  The  potential  targets  of 
opportunity  for  ET  include  collective  tactical  training 
requirements.  For  example,  the  Systems  Training  Plan  for  the  Main 
Battle  Tank  Block  III  states  "embedded  training  will  include  one 
or  more  stand-alone  modules  for  force  on  force  simulation"  (U.s. 
Army  Armor  School,  1989b) . 

ET  may  take  three  forms  as  described  in  Witmer  and  Knerr 
(1991);  fully  embedded,  appended,  and  umbilical.  In  the  fully 
embedded  version,  all  training  features  except  for  certain 
software  and  courseware  are  contained  within  the  prime  weapon 
system.  Appended  ET  can  be  installed  or  attached  temporarily  to 
the  prime  weapon  system  to  support  training.  Umbilical  ET 
involves  temporary  connections  to  external  components  to  support 
training.  The  application  of  ET  to  collective  training  above 
crew  level  will  involve  a  form  of  appended  or  umbilical  ET, 
because  the  various  vehicles  must  exchange  information  with  one 
another,  A  rough  translation  of  the  SIMNET  model  to  the  ET  model 
would  involve  replacing  the  vehicle  simulators  with  prime  weapon 
systems  that  include  graphics  generators.  The  vehicles  would 
then  be  linked  to  allow  data  broadcast  from  one  vehicle  to  be 
picked  up  by  other  vehicles. 

The  capability  to  include  an  automated  feedback  system 
is  a  major  variable  in  determining  if  and  how  ET  might  be  applied 
to  a  particular  training  requirement  in  a  cost-effective  manner 
(Oberlin,  1987;  Strassel,  Dyer,  Roth,  Alderman,  and  Finley, 

1988) .  In  many  cases,  the  requirement  to  include  training 
software  and  an  automated  performance  measurement  system  withi.n 
the  prime  weapon  system  would  overload  the  weapon  systems  organic 
computers.  However,  the  UPAS  system  is  capable  of  being  linked 
to  a  network  of  vehicles  and  providing  its  own  computing  power. 
Therefore,  the  UPAS  might  be  applied  directly  to  future  attempts 
to  embed  collective  training. 

Research  on  SIMNET  Practice  and  Feedback  Variables 

The  fact  that  SIMNET  is  an  electronic  battlefield  means  that 
data  are  available  to  support  training  research.  However, 
translating  this  wealth  of  information  to  a  format  that  supports 
documentation  of  training  (practice  and  feedback)  and  measurement 
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of  unit  performance  is  a  substantial  task,  comparable  to  that  of 
preparing  and  organizing  NTC  data  for  analysis  (McFann,  Hiller, 
and  McCluskey,  1990)  .  The  UPAS  can  be  used  to  support  training 
research,  as  well  as  supporting  AARs. 

Thorpe  (1988)  provides  a  summary  of  evidence  that  SIMNET  is  an 
effective  training  method.  TEXCOM  (1990)  examined  the  transfer 
of  SIMNET  training  to  field  testing  of  platoons  on  collective 
tasks  from  MTP  documents  and  concluded  that  "both  tank  and 
mechanized  infantry  units  improved  in  overall  performance  after 
receiving  SIMNET  training.''  Bessemer  (1990)  provided  evidence 
that  the  addition  of  SIMNET  training  to  the  program  of 
instruction  for  Armor  Officer  Basic  (AOB)  students  had  a 
beneficial  effect  in  terms  of  leader  performance  evaluations 
during  subsequent  field  exercises.  None  of  these  efforts  was 
able  to  use  appropriate  control  groups,  provide  information  about 
the  specific  collective  influenced  by  SIlvlNET,  or  document 
practice  and  feedback  variables  in  detail. 

The  SIMNET  training  transfer  research  was  accomplished  in 
order  to  decide  whether  the  relatively  low  level  of  fidelity  of 
certain  aspects  of  SIMNET  prevent  transfer  to  field  training 
(Kerins,  Atwood,  and  Root,  1990).  However,  there  is  a  need  to 
identify  and  assess  the  impact  of  practice  and  feedback  variables 
that  influence  the  transfer  of  SIMNET  training.  Two  reports  have 
examined  variables  that  might  influence  SIMNET  training. 

Bessemer  (1990)  concluded  that  the  effects  of  SIMNET  training  on 
subsequent  leader  evaluations  increased  with  the  instructor's 
experience  in  using  SIMNET.  He  also  noted  that  the  conduct  of 
AARs  tended  to  improve  over  time.  A  later  effort  by  Shlechter, 
Bessemer,  and  Kolosh  (1991)  looked  at  the  effects  of  the  role 
played  during  SIMNET  training  on  subsequent  evaluation  of  leader 
performance  and  found  evidence  that  students  serving  in  the  roles 
of  platoon  leader  or  platoon  sergeant  gained  more  benefits  from 
SIMNET  training  than  did  those  students  that  served  only  as 
vehicle  commanders,  gunners,  drivers,  and  loaders.  As  in  the 
case  of  Bessemer's  work,  these  researchers  found  evidence  that 
the  beneficial  effects  of  SIMNET  training  were  enhanced  as  the 
instructors  collectively  gained  more  experience  with  SIMNET. 

These  efforts  address  only  a  few  of  the  many  practice  and 
feedback  variables  that  need  to  be  examined  in  the  SIMNET 
environment. 

The  UPAS  has  several  features  that  are  expected  to 
facilitate  future  SIMNET  training  research.  First,  the  UPAS 
has  the  capability  to  generate  a  wide  variety  of  displays  for 
providing  feedback  to  units,  allowing  researchers  to  vary  and 
document  feedback  in  a  systematic  fashion.  Second,  the  UPAS 
provides  a  means  of  documenting  the  practice  actually  received  on 
specific  collective  tasks  during  SIMNET  exercises.  Third,  the 
UPAS  contains  a  variety  of  flexible  tools  that  can  be  used  to 
measure  unit  performance. 
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